There has been emerging trend for accelerated databases where extreme database scalability is achieved by leveraging power of massively large number of computational nodes in efficient processing of analytical database workload. Such a system relies heavily on the data availability and correctness.
Data may be stored in a large cluster of nodes to take advantage of the aggregate memory and processing power of a large number of nodes. Clusters are available that operate on data that is stored in memory. Such clusters may lack persistent storage and access to transaction logs, and may relying instead on a traditional relational database management server (RDBMS) to be ACID (Atomicity, Consistency, Isolation, Durability) compliant, as expected in standard database systems. The cluster assists the RDBMS in processing the workload.
Failures, either in the RDBMS or the cluster, have a severe impact because they can lead to either data loss or data inconsistency. Several types of failure may affect transactional consistency. For example, when the RDBMS fails, change records used to maintain transactional consistency may be lost. When a node storing distributed data fails, the data must be completely reloaded from a source database. Typically, to prevent inconsistencies, the data must be locked or otherwise made unavailable during reloading.
Existing change propagation methods for synchronizing data between a source and a destination include log-based methods and load-based methods. Log-based methods replay, at the destination, transactional log records for operations performed on the source. For example, the transactions may be re-executed and applied at the destination in the order in which the transactions occurred. Load-based methods periodically reload one or more tables from the source to the destination.
The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section.